Add MaskRCNN model as a test for the wrapper approach with TF model garden code #754
Conversation
Mostly copy/pasted from internal roadmap doc (with some modifications).
…arden code. 1. The current class only contains the model building logic. 2. Most of the tunable params are currently backed in the function, and not exposed to use yet. 3. Verify the model building logic in the test. (will need tfm pip deps to work) Next step: 1. Configure the input parsing logic for coco, and also anchor generation. 2. Mimic the training loop logic as model.train_step/eval_step 3. Train model e2e.
qlzh727
requested review from
fchollet,
LukeWood,
martin-gorner and
tanzhenyu
|
Comments from the #753:
|
|
Relevant ticket. #623 |
There was a problem hiding this comment.
Thank you for this first attempt.
To bring this further, could you be able to test it actually training, with compile and fit working ?
I have created a starter MaskRCNN notebook here with a small bur real dataset that can be used for further testing:
https://source.cloud.google.com/cloudml-demo-martin/martin-kerascv-test/+/master:Mask_RCNN_test.ipynb
I have not been able to test very far though.
| classes=91, # default value for coco | ||
| input_size=[512, 512, 3], | ||
| backbone="resnet", | ||
| backbone_weights=None, |
There was a problem hiding this comment.
I assume "resnet" here refers to Model Gaden's ResNet ?
I tried passing "spinenet_mobile" but got an error.
There was a problem hiding this comment.
Currently supported_premade_backbone = ['resnet', 'spinenet', 'mobilenet']
There was a problem hiding this comment.
I tried "spinenet" but it errors out:
decoder = tfm.vision.decoders.FPN(
--> 129 input_specs=backbone.output_specs, # Might need to check for custom backbones
130 min_level=min_level,
131 max_level=max_level,
AttributeError: 'str' object has no attribute 'output_specs'
There was a problem hiding this comment.
Yes these backends are TODO. Only resnet now:
| ) | ||
| images = tf.random.uniform(shape=(2, 512, 512, 3)) | ||
| image_shape = tf.constant([[512, 512], [512, 512]]) | ||
| anchor_generator = tfm.vision.anchor.build_anchor_generator( |
There was a problem hiding this comment.
To align with keras_cv, can a default anchor generator be provided when the user instantiates the model?
There was a problem hiding this comment.
Also, for dome reason, tfm.vision.anchor does not work for me. Maybe the symbol is not exported ?
| [[200, 200, 300, 300], [300, 300, 400, 400]]]) | ||
| gt_classes = tf.constant([[1, 2], [3, 4]]) | ||
| gt_masks = tf.constant(1, shape=[2, 2, 512, 512]) | ||
| output = mask_rcnn_model( |
There was a problem hiding this comment.
This is a single call. Does .fit() work?
| def test_maskrcnn_construction(self): | ||
| mask_rcnn_model = keras_cv.models.mask_rcnn( | ||
| classes=91, # default value for coco | ||
| input_size=[512, 512, 3], |
There was a problem hiding this comment.
Is the input_size parameter necessary ?
Usually in Keras models, this gets built from the data.
|
Open questions:
Note: the goal here is to wrap the Model Graden model but not the Model Garden data pipeline code. Users can use Keras preprocessing layers to format a data feed appropriately and we will provide example code in the accompanying templates. |
|
Should we close this as we most likely will add MaskRCNN on top of existing FasterRCNN? |
yeah I think so. |
|
Discussed offline, it seems we all agree to close it for now. |
Note that this PR is more for a demo and not intend to be submitted at the moment.
What does this PR do?
Add MaskRCNN model as a test for the wrapper approach with TF model garden code.
Next step:
Before submitting
Pull Request section?
to it if that's the case.
Who can review?